The trend in enterprise or distributed computing, and in information processing in general, is toward sharing or integration of data sources between a variety of otherwise disparate applications. The traditional data processing architecture utilizes large-scale database systems, rarely if ever linked together by any intelligent means, that are used to serve up data to independent, focused, and often heavily customized applications. As a result of the historical legacy in computer architecture and the prevailing use of traditional methods of data processing, much useful data is locked in application “stove pipes”, creating islands of information whose full potential is not fully realizable.
Integrating these islands of information is a complex and often costly exercise. Typically such projects require the skills of experienced data experts, and the need for expensive one-off data interfaces for each individual project. The result is a complex information technology (IT) environment, with continuously increasing maintenance needs and costs.
To address these issues, some system providers have turned to data warehousing techniques to better share and facilitate data exchange between an enterprise suite of applications. However, data warehousing alone cannot provide the answer—in most cases the data becomes stale too rapidly for meaningful or reliable integration. Data warehouses are of most use in decision support systems that rely on the ability to quickly scan a database or data repository and base a decision-making process on, and are of much less when the data changes rapidly. Enterprise Application Integration (EAI) systems have attempted to bring data warehousing benefits to the application level, but they typically demand procedural, synchronous programming that is highly optimized for tightly coupled applications. However, no mechanism currently exists for reliably and tightly coupling the wide variety of applications with the underlying data in a fully integrated manner.